Systems and methods involving features of adaptive and/or autonomous traffic control

ABSTRACT

Systems and method are disclosed for adaptive and/or autonomous traffic control. In one illustrative implementation, there is provided a method for processing traffic information. Moreover, the method may include receiving data regarding travel of vehicles associated with an intersection, using neural network technology to recognize types and/or states of traffic, and using the neural network technology to process/determine/memorize optimal traffic flow decisions as a function of experience information. Exemplary implementations may also include using the neural network technology to achieve efficient traffic flow via recognition of the optimal traffic flow decisions.

CROSS REFERENCE TO RELATED APPLICATIONS

This is a continuation of application Ser. No. 13/369,233, filed Feb. 8,2012, now Pat. No. 8,825,350, which claims benefit/priority ofprovisional application No. 61/562,607, filed Nov. 22, 2011, this isalso a continuation (bypass) of PCT application No. PCT/US14/49839,filed Aug. 6, 2014, all of which are incorporated herein by reference inentirety.

BACKGROUND

1. Field

The present innovations relate generally to traffic control, and, more,specifically, to systems and method involving adaptive and/or autonomoustraffic control.

2. Description of Related Information

Neural network technologies have been in development for decades wherereal-time computing problems are solved by using software or circuitrywhich emulates the brain's function. The human brain contains roughly100 billion neurons. About 300 million neurons are dedicated to thevisual cortex, just one of several sensory input sources to the brain.Small to medium scale artificial intelligence systems using neuralnetwork technology have been used successfully in many real-worldapplications such as pattern recognition for industrial process sortingor quality control functions and real-time navigation and collisionavoidance systems, with results meeting or exceeding human capabilities.For example, challenges sponsored by DARPA have taken place whereautonomous vehicles employing neural network technology havesuccessfully navigated across vast distances of hazardous desert terrainor through urban courses.

Human brains and artificial neural networks both store memorized visualimages, other sensory input, and related sequences thereof to makereal-time predictions, i.e. decisions, from those sensory inputs. Aswith other well-adapted animal species having brains smaller than thoseof humans, artificial neural network systems of modest size can performwell on tasks calibrated to the size of “brain”. Traffic lightcontrollers perform a critical task in modern society and represent atechnical challenge well within capabilities of emerging neural networktechnology.

Practical solid-state devices currently in production provide economicsolutions to many real-world problems. Current and proposed solid-statetechnologies, such as flash memory and memristor devices, offer veryhigh density analog nonvolatile storage elements well-suited toconstructing high-density solid-state analog neural network devices.Typical flash memory arrays of 4 billion transistors or more in sizecould yield neural network devices with 4 million or more neurons. Whilethe human brain is capable of incredibly complex pattern recognition andprediction, artificial intelligence systems with far fewer neurons canaccomplish important real world tasks. Current implementations ofsolid-state neural network devices have bridged the gap between humanand artificial neural network capacities with techniques such asreducing size of input data streams with integrated digital signalprocessing (DSP) modules.

Artificial analog neural networks have recognition engines typicallyemploy K-Nearest Neighbor (KNN) or radial basis function (RBF) nonlinearclassifiers or both. While KNN classification is useful in applicationsseeking merely the closest match to a recognized pattern, RBFclassification is particularly useful in traffic control applications byvirtue of its “yes, no, or uncertain” output states. Current solid-statedevices, in addition to having such DSP and classification modules, canalso be interconnected for scaling to larger, multi-level neuralnetworks. As such, aspects of the present innovations may be implementedwith existing technology or with future devices having larger neuroncapacity or with future, fully integrated solid-state devices having allthe capability herein described.

Certain advantages of the systems and methods herein are obvious toreaders having personal experience with automobile travel. Among otherthings, current vehicle sensors provide inadequate recognition ofincoming traffic, forcing traffic to stop before traffic signal lightchanges are effected. Or, as will be shown, sensors intended to provideadvance traffic flow information cannot fully comprehend driverintentions, resulting in incorrect decisions by current traffic lightcontrol systems. While hybrid automotive technology improves efficiencyby capturing energy otherwise lost when vehicles are stopped, it is evenmore efficient to regulate traffic flow to maximize overall throughput,minimizing cumulative vehicle wait times. Vehicle wait times equalpassenger wait times, such that improved traffic flow yields bothimprovements in overall fuel economy and increased driver productivity.Innovations herein include systems capable of providing comprehensivetraffic control system functionality, such as full implementation ofvarious features and aspects.

Further aspects consistent with the present innovations relate tooverall system performance due to real-time, parallel recognition andtraffic flow decision selection by the neural network. As with a humanobserver or traffic officer, a comprehensive overall picture of trafficflow needs yields an immediate decision for optimal traffic flow throughan intersection. As such, according to some implementations,intermediate hierarchical layers of the neural network aggregate such anoverall picture from which a specific, optimal traffic light sequence isselected. Performance of significantly higher magnitude than existingsystems may be achieved via a fully integrated solid-state deviceimplementation of the system, surpassing that of algorithmicimplementations using digital processors.

OVERVIEW OF VARIOUS ASPECTS

Aspects of the present innovations may include or involve systems andmethods for providing high accuracy recognition of all types of trafficrequiring passage through an intersection with means to add and improverecognition of such traffic, to optimize the prioritized flow of trafficthrough the intersection, and/or to adapt to new traffic types,technologies, priorities, and/or traffic flow needs. In someimplementations, neural network arrays may be used to store recognizedtraffic objects and traffic flow patterns thereof, enabling autonomoustraffic flow control by the local traffic light controller without theuse of system-wide central synchronization or static, predetermineddigital control algorithms.

Drawbacks may be overcome in accordance with aspects the presentinnovations via provision of improved control systems and methods fortraffic lights which makes significantly better decisions regarding thetype and intentions of incoming traffic while reducing both initialinstallation and long-term upgrade costs by means of artificialintelligence in the form of a neural network based controller havingreal-time adaptive learning ability and upgradability.

Thus, advantages relating to one or more aspects of the presentinnovations include having real-time adaptive capabilities which enableoptimized traffic flow control superior to fixed algorithms of digitalcontrol systems.

According to some implementations, further advantages may be achievedvia autonomous operation without external human intervention orfixed-timing synchronization with nearby systems.

Advantages related to other implementations include providing superiordetection of, signal control for, and, hence, superior traffic flowperformance for all types of traffic incoming to the autonomous signallight.

Other further advantages of the present innovations relate to theprovision of a design and methodology enabling lowest-cost, highperformance reliable, solid state implementations of the capabilitiesset forth herein.

Still further advantages of systems and method herein are achieved as afunction of the size of emerging low-cost neural network arraysabsorbing functions of digital logic for improved performance atlowest-cost by eliminating digital component costs and processingdelays.

Further aspects of the present innovations may be seen in connectionwith additional implementations set forth herein and/or provided via theappended specification, drawings and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other items, features, and advantages of the inventionswill be better understood by reading the following more particulardescription of the innovations in conjunction with the accompanyingdrawings wherein:

FIG. 1 is a global block diagram of an illustrative system, consistentwith one or more aspects of the innovations herein.

FIG. 2 is a diagram illustrating one common traffic light arrangementand a sensor array, consistent with one or more aspects of theinnovations herein.

FIG. 3 is a diagram illustrating an existing typical signal lightconfiguration which illustrates some limitations of prior systems.

FIG. 4 is a diagram showing an illustrative traffic signal lightsequence reflecting useful functional aspects consistent with one ormore aspects of the present innovations.

FIG. 5 is a diagram showing another illustrative traffic signal lightsequence reflecting useful functional aspects consistent with one ormore aspects of the present innovations.

FIG. 6 is a diagram detailing traffic object classification, traffictype classification, individual and group traffic object current stateclassification, and broader traffic system network input aggregation, asmay be used e.g. in the training algorithms in the present neuralnetwork systems consistent with one or more aspects of the presentinnovations.

FIG. 7 is a flowchart depicting an illustrative process for initialsystem configuration consistent with one or more aspects of the presentinnovations.

FIG. 8 is a flowchart depicting an illustrative process flow for neuralnetwork array training consistent with one or more aspects of thepresent innovations.

FIG. 9 is a flowchart depicting startup and ongoing operation aspects ofan illustrative system, consistent with one or more aspects of theinnovations herein.

FIGS. 10A-10B are a diagram and a flowchart, respectively, ofillustrative higher-level recognition processing as performed by aNeural Network, consistent with one or more aspects of the innovationsherein.

DETAILED DESCRIPTION OF ILLUSTRATIVE IMPLEMENTATIONS

Reference will now be made in detail to the inventions, examples ofwhich are illustrated in the accompanying drawings. The implementationsset forth in the following description do not represent all embodimentsconsistent with the claimed inventions. Instead, they are merely someexamples consistent with certain aspects related to the innovationsherein. Wherever possible, the same reference numbers will be usedthroughout the drawings to refer to the same or like parts.

FIG. 1 shows a system block diagram inclusive of all major functionalelements along with primary input sources and state/control outputdestinations. According to some implementations, the elements encircledby the dashed line 102 represent a full set of components/functionalityby which aspects of the present innovations can be implemented in one ofseveral scales of integration. Here, some current technologies may beutilized to achieve multi-chip solid-state implementations, whileemerging high-density solid-state neural network devices can yieldhigher integration levels, all the way to future implementationsachievable in monolithic solid-state devices.

Systems herein may be implemented with a local input/output [I/O] module104 that transmits uni- or bi-directional signal data or control signalsto or from local sensors and devices of a traffic control system. Module104 may include an analog to digital (A/D) converter such that allinputs to the system can be represented as digital values includingthose from analog sensors. A multiplicity of system clocks 106 may alsobe included to synchronize system processes. For example, the coreprocessing elements of the microprocessor 108 and digital signalprocessing [DSP] elements 110 may operate at the highest frequency whilethe main memory (112, 114) and system data bus cycles 116 may be slower,typically some multiple of those core processor frequencies. In turn,the real-time image capture frequency may generally match that of theneural network recognition engine, or classifier 118, cycle time, lowerthan that of those previous processes. There may be multiples of thistype of clock or they may be gated to specific registers of the localI/O module as a function of the data set or “context” for which aspecific set of neurons is trained to recognize. For example, the neuralnetwork recognizes traffic from all directions incoming to theintersection, so that different contexts may be used to distinguishamong traffic flows, especially in the case where a top view cameraprovides input vectors for more than one direction. Other contexts maybe desired where sensor arrays are unique to different trafficdirections. Hence at a typical intersection, the neural network may run4 recognition cycles, one from each direction, for each data capturecycle of the sensor arrays. Lastly, some programmable count of systemclocks is used to vary the timing of signal light via programmable timermodule 120 input to the programmable sequencer logic 122. Additionalclock sequences may be used to synchronize I/O with independent externalsystems such as radio communication with standardized “high priority”signal control protocols for emergency response vehicles [ERVs] or highoccupancy vehicles [HOVs]. An external power supply and switching relays124 directed by the output of the programmable sequencer logic 122 drivethe individual lens lights of the traffic signal lights 126.

Digital signal processing (DSP) modules 110 may be included, as needed,and may reduce the size of video and other sensory input to match thesize of data input to the registers of the neural network array. Digitaldomain memory may include one or both of reprogrammable nonvolatilememory 112, such as flash memory, for firmware storage and dynamicworking memory 114, such as static RAM. Firmware may be configured tocontain control code for the microprocessor's system supervisory code,training software for optimizing performance of the neural networkarray, and/or system control data parameters such as data values writteninto the programmable timer module 120. Classifier logic 118 may beincluded for the output of the neural network array 128 and, in thisapplication, may be of the radial basis function (RBF) type best suitedfor traffic object recognition and flow control decision-making. Theneural network training process may deploy additional neurons as newtraining examples are provided, simultaneously adjusting the distance,or “influence field,” of objects to be recognized relative to theexample such that each neuron recognizes only the specific example forwhich it is trained. A distance of zero means an exact match between newinput data and stored vectors. A selected set of neurons assigned to thesame “context” may define a “recognition engine” for a predeterminedclass of learned objects, patterns, or decisions. Within a givencontext, training specific to a neuron or a subset of that context's setof neurons are assigned a “category” to differentiate among uniqueobjects, patterns, or decisions within that class. A number of differentneurons are trained to produce recognition results which vary with thenumber of objects to be learned and required accuracy of the recognitionresults.

In some illustrative implementations, valid RBF classifier outputs maybe sorted into 3 categories: “Identified”, certain with learneddistances, i.e. the neuron's influence field, for one or more neuronsall belonging to a specific category, which is designated here as “Match1”; “Uncertain,” a possible match with stored vectors of 2 or moreneurons of different categories, designated as “Match 2”; or “Unknown,”i.e. an input vector which is unrecognized by all currently trainedneurons in that context. Unknown results will also occur when there isno activity in the traffic light area, i.e. when there is nothing torecognize. Match 1 and Match 2 are associated flags which can be used totrigger interrupts or polling routines to the microprocessor to initiatechanges in traffic light control sequence or timing. The programmableinterconnect logic 130 is used to gate which matches, primarily highestlevel traffic-optimizing decisions, the microprocessor acts upon.

Neurons can be trained to recognize any vector presented to them.Generally, these vectors are presented to a majority or all the neuronsof a hierarchical level simultaneously on the system data bus which isthe case in recognizing individual traffic objects from video input.Just as with the human brain, however, other neurons can be trained torecognize sequences. Configurable I/O 134 enables the training ofsequences. Classifier logic output from specific neurons may be sampledoutputs from various low-level neurons at a programmable rate, i.e. arate established as a function of one or more system clocks, and may bestored in shift registers in the configurable I/O. As such, a sequenceof classifier results from those neurons can be assembled into a vectorthat may be used as input to neurons in higher layers of the neuralnetwork hierarchy. Neuron input vectors may be loaded as a sequence ofbytes up to the maximum vector size of the neuron implementation.Certain higher level neurons may be wired via programmable interconnectlogic to recognize traffic states from lower level neurons, such as incases where relative speed or grouping of objects is being recognized.The configurable I/O may include data buffers for such higher levelneurons for assembly of output data from multiple lower-level neurons inparallel.

A common method for training neural network arrays is back propagationwherein weightings individual inputs to specific neuron are determinedby an iterative process which yields an output for a recognition resultor decision, defined by a prototype example of the correct recognitionof an object, environmental state, or decision. Training data sets, bothvalid vectors to be positively recognized and counter examples to berejected, may be presented to the neural network in the trainingprocess, at the end of which an appropriate number of neurons withappropriate influence fields are trained to recognize the objects,patterns, and decisions required by the application. Thus the design andprioritized weighting of the training data sets are critical to systemperformance.

Both fuzzy logic and neural network based systems have previously beenproposed as the decision-making engine for traffic light controllers.Similarly, neural network training algorithms themselves are welldocumented and therefore not discussed here. However, systems andmethods herein include system-level capabilities related to deployingvery large neural network arrays. Among other things, systems and methodwith such large capacity may involve or support one or more of: thetraining specificity of individual lower-level neurons; additionalclasses of recognition available to enhance system capabilities usinghigher-level neural network layers; and/or an architecture supportingthe customization, optimization, and low cost upgrade capability of thesystem. Another significant capability offered by the systems andmethods herein is the ability to re-train individual recognition enginesin real time with full system functionality by reserving spare neuronsfor this purpose.

Considerations, features and functionality related to decision-makingpriority weightings may be salient aspects of the current innovations,as described in part in subsequent drawings and tables. An advantage ofthe neural network array, according to certain embodiments, is theparallel processing nature such that a particular input vector isprocessed by all neurons in that hierarchical level simultaneously sothat unique recognition results appear within one recognition enginecycle, far faster than otherwise achievable through digitalcomputational methods. Such an immediate recognition by the lowest orderelements of the neural network array allows immediate classification ofa traffic object type. Similarly, immediate recognition of a trafficflow pattern in a higher level of the neural network array yieldsrecognition of particular traffic situations through aggregation of thecurrent state of traffic objects entering or present within the relevanttraffic control range of the specific traffic signal light. Otherintermediate levels of neural network array may be trained to recognizeunique individual features, grouped traffic objects, and nearby incomingtraffic relative to the needs of the local traffic area and any widercentral traffic network system. Highest levels of the neural networkhierarchy may receive outputs from selected lower-level neurons gated bythe programmable interconnect logic 130 which are used to selectspecific, optimal traffic flow decisions when the appropriately trainedneural network generates a match between current conditions with aspecific decision. The decision is the firing, or “Match1”classification, of one or more highest level neurons with which aspecific programmable sequencer logic subsequence entry point andprogrammable timer module data set is associated. The microprocessor 108may retrieve these sequencer entry points and timer module data fromsystem control data stored in firmware 112 and may write these data intoinput registers of the programmable sequencer logic 112 and programmabletimer module 110. Alternatively in some straightforward systems, theprogrammable timer and sequencer may be implemented as software routineswith output results stored in microprocessor registers which are writtenout to simple external hardware latches connected to signal light powersupply and switches circuitry (124) at the expense of some performanceand functionality.

Prior neural network architectures provide parallel vector input;polling methods to identify next available, or ready-to-learn, neurons;and signal propagation between neurons are all useful in the presentinnovations in a given hierarchical layer. Early implementations may usecombinations of smaller neural network arrays to achieve the minimumrequired hierarchy levels and sizes, but such approaches cannot have theupgrade flexibility of a very large neuron array which is infinitelyreconfigurable via programmable logic, such as with the systems andmethods herein. Direct programmable interconnect(s) consistent with thepresent innovations may improve performance by enabling all networkhierarchical levels to operate in parallel.

Local inputs include sensor arrays 140, which may include various newertypes of sensors, at the traffic light providing the traffic signallight point of view, pre-existing inputs, such as pedestrian crosswalkbuttons and weight- or inductive loop vehicle sensors 142, and/or otherdata capture devices providing different perspectives, such as an offsetvideo camera 144 which may also serve as a traffic violation recordingdevice.

The local I/O module 104 may be configured to handle both digital andanalog signals and has sufficient spare channels which may be assignedto future devices to support the desired upgradability. Some of thesechannels are bidirectional so that control commands can be sent back tospecific devices such as a video recording module.

The system bus 116 is bidirectional by definition, and, as shown in FIG.1, may interconnect the microprocessor 108 and digital memory (112, 114)with the communication interface 132, the programmable interconnectlogic 130, the neuron classifier logic outputs 118, programmable timermodule 120, and the programmable sequencer logic 122.

The programmable interconnect logic 130 may be configured to allocatethe required number of neurons to each of two or more hierarchicallevels and to configure the interconnections between them. Theprogrammability allows subsequent upgrade to reconfigure the neuronallocations and hierarchy as improved traffic control methods aredeveloped without requiring hardware changes. Another function that maybe programmed into the programmable interconnect logic 130 is to gatewhich “matches” detected from the neural network array are gated to themicroprocessor 108 for required action in changing traffic lightsequence and timing.

Similarly, the programmable sequencer logic 122 may be configured atinitial traffic light installation customized for traffic lane andsignal light configurations specific to each orthogonal intersectionapproach. Example sequences are discussed in subsequent figures. Alongwith the sequences, default initial values for the programmable timermodule parameters are configured upon initial signal light installation.This provides for “FailSafe” operation in case of sensor array device,microprocessor control, or neural network array function failure.Traffic lights consistent with the present innovations may continue tooperate in the same fashion as many existing traffic signal lightsaccording to the timings and signal light sequences of the defaultconditions programs into the programmable timer module 120 andprogrammable sequencer logic 122.

As noted previously regarding other features of the innovations here,the programmability of the neuron array interconnect logic, the timermodule, and a sequencer logic provide upgradability without any hardwarechanges to the basic signal light controller.

Current and next state table of the programmable sequencer logic may beused to communicate status to similar adjacent traffic light controllers150 for traffic flow optimization as described in the presentinnovations or to communicate with incoming traffic object vehiclesinteractively.

The communication interface module 132 may be configurable to providevaried functionality. It may provide an interface for all remote inputsand outputs. For example, as described previously, the current state andstatus of traffic objects and groups of traffic objects of nearbysimilar traffic light controllers 150 can be used to optimize trafficflow control at this particular traffic signal light. Likewise, theoutput of current state information of both the signal light sequencerand higher-level group traffic object position and velocity statusinformation is useful to similar adjacent traffic light controllers foroptimization traffic control at each traffic signal light location.Therefore, each traffic light controller operates in completelyautonomous fashion while being coordinated with other similarcontrollers in the larger traffic control network system. Instead offixed signal control coordination across the array of traffic controlnetwork signal lights, each traffic signal light controller may beconfigured with components for receiving and processing informationregarding awareness of the multiplicity of local and nearby traffic flowstatus and for making optimal, autonomous decisions for traffic controlsignaling for the specific site.

The communication interface module 132 may be arranged as the conduit toa remote command center 152 for effecting local system upgrades orsystem-wide control of the local site. In some implementations, centraltraffic flow control algorithms may indicate optimal system-wide trafficcontrol requires forcing of local traffic light sequences intoparticular states. Thus, this communication interface from the centralnetwork wide command center allows preemption of local system by centralcommand. Normal autonomous operation may be optimized by initial set upand periodic updates, as required, from the remote command center 152.These may include changes to configurations of sensors to specificneurons via retraining or training of new neurons in the lowesthierarchical levels; changes to configurations of neuron to neuronconnections via updates to the programmable interconnect logic 130; thedownload of updated, externally acquired training into RAM 114 to beprogrammed directly into specific neurons 128; the download of newtraining algorithms to system firmware 112; the initiation of trainingsessions at this specific intersection using downloaded trainingalgorithms; and/or changes to sequences and timing sets written into theprogrammable sequencer logic 122 and the firmware 112 control data forthe programmable timer module 120, among other things.

An external radio antenna array may be included, and may be designed tocapture frequencies specific to all relevant communication protocolsthat may be needed by the controller, including broadband datacom (e.g.WiFi) and frequencies used for standardized ERV and HOV communication.Similarly, the system's communication interface module may includedigital logic for required standardized datacom interfaces such as WiFi.The present systems may also be configured to free up the microprocessorfrom the task of monitoring specialized signals. As mentionedpreviously, one set of system clock timing may be used to synchronizedata capture from the radio antenna arrays with the timings ofspecialized signals such as ERV and HOV requests. The data may be feddirectly into higher level than neurons trained to detect the uniquesequence of signal preemption requests transmitted by ERVs or HOVs.Recognition of such a unique signal by neural network may generate aunique interrupt request to the microprocessor, which then begins anyrequired authentication process for validating and executing thepreemption request. Similarly, any combination of sensory input theneural network is trained to recognize can be used to generateinterrupts triggering specialized control sequences supervised by themicroprocessor 108.

The local nonvolatile digital memory 112 of the system may storetraining algorithm software for optimization of the neural networklearning. As previously noted, the communications interface 132 mayrelay commands from a remote command center to enable continued learningor re-optimization of traffic flow decision-making as required bychanges in local traffic flow conditions. Through the same interface,improved training algorithm may be downloaded at any time to benonvolatile digital memory 112. Further, the training itself, acquiredthrough simulation by similar controllers at remote sites, can bedownloaded via the microprocessor 108 directly to the neural networkarray in the form of specific analog weightings of specific neurons inthe local array. Similarly, traffic flow control decisions learned atremote simulation sites may be downloaded as analog weightings of theappropriate neurons in higher levels of the neural network arrayhierarchy. As such, systems and methods herein may be upgraded at anypoint in time based on new learning acquired at the local site orlearning which is uploaded from remote location training withouthardware changes to the local controller.

The basic firmware executed by the microprocessor 108 may initiatedefault traffic light control sequences programmed into the programmablesequencer 122 to provide failsafe operation in the case of sensory arrayor neural network control failure. Pre-existing sensors, such asconventional vehicle sensors (e.g., inductive/loop sensors, weightsensors, etc.), could even be monitored by the microprocessor in theevent of such failures such that operation identical to that of current,conventional traffic lights may be effected in FailSafe mode. SuchFailSafe mode(s) may also used for basic startup operation or wheneverno traffic activity is detected.

FIG. 2 shows a typical traffic light configuration for a common two-lanehighway with a dedicated left turn lane with a traffic sensor array 202as may be used consistent with the present innovations. The conventionaltraffic signal light 204 in this case, as shown, has the standardizedred, yellow, and green signal lights with the green left turn indicatorbelow. Other conventional components are weight- or inductive loopvehicle sensors 206 and mechanical switches 208 at the light pole forpedestrian crossing. Weight sensors could be physical weight measurementdevices but most common conventional implementations are “inductiveloops” with simple threshold detection of traffic objects containingferro-magnetic materials being sensed “digitally”, i.e. as being presentor absent. These are often miscalibrated so that some traffic objectsare missed and obviously cannot detect traffic lacking conductive (e.g.ferrous) materials. If the analog values of inductance are senseddirectly and presented as a range of values, the inductance may providean alternative means of sensing weight as approximated by the amount ofconductive material detected at the sensor. Other symbols included inthe drawing are for conceptual understanding only, as opposed toproviding detailed mechanical representations. Video cameras on bothleft 210 and right 212 extremes of the horizontal suspension arm may beincluded and may provide ideal parallax video input for recognition oftraffic object position and distance. Alternatively, a single frontalvideo camera may be sufficient for most purposes. Another camera labeled“top view video” 214 can provide additional data input for recognizingpedestrian and/or potential traffic light violators. With sufficientlywide angle, such a camera could capture pedestrian traffic for alldirections and possibly obviate the need for other video cameras. Asdescribed previously, a radio antenna array 204 is included to captureall standardized datacom and specialized radio signal capture andtransmission to and from the traffic light. As an alternative to radiosignaling for specialized preemptive signal control, specific audiopatterns such as sirens from ERV's can be sensed as an additional datatype recognizable by neural network, as represented by the symbol for anaudio sensor 216 in FIG. 2. The present systems may be designed withsufficient spare neuron capacity to accommodate additional sensory inputfrom various sources as represented by another FIG. 2 symbol indicatingcapture of infrared [IR] or other electromagnetic spectra 218. Thecapture of additional spectra may be useful for detecting higher-leveltraffic characteristics such as estimating the number of passengers inany given vehicle. Similarly, the deployment of a radar unit 220 whichcan provide proven vehicle distance and velocity data may be desired,though it should be noted, as described elsewhere, that this added costof a radar unit may be obviated by the combination simpler sensory inputand higher-level neural network recognition of relative vehicle positionand velocity. Aside from speed and distance calculations, the radar unitmay be useful for improved traffic object classification. Along with IRor other spectra sensor input, unique radar signatures returned byincoming traffic objects may be useful to classify them in dark or badweather conditions. Multiple neurons may be assigned to each unique“match” for a traffic object class recognized by different sensors indifferent conditions. Thorough experimentation will yield the optimalconfiguration of sensor arrays and neural network capacity in regards toperformance versus cost for any given traffic control environment.

Another symbol in FIG. 2 indicates a rearview video camera location 222.This is commonly used for additional data capture for the purposes oftraffic light violation detection and recording, as described in priorart. Another purpose for such a video camera installation is to handlethe case where visibility of the local traffic light area is obstructedby raised road surface areas or other physical barriers. In these cases,improved sensory input to the traffic light controller yields greatertraffic flow performance at lower cost than conventional sensors such asroadway-embedded inductive sensors.

Of interest to certain aspects of the present innovations may beutilization of any and all video capture devices being superior in manycases to conventional roadway-embedded inductive sensors 206. Thisadvantage is obvious to motorcyclists and bicyclists, since conventionalsensors often fail to detect their presence. The current innovations arefar superior in such regard. Neural network recognition of learned videoimagery for motorcyclists and bicyclists provide traffic control systemhigh confidence recognition of these traffic light users. Whilecontemporary thinking may continue to assign lowest priority to thistype of traffic, the ongoing efforts to improve overall fuel economywill motivate the desire to provide most efficient traffic control forthis type of traffic. Superior, expanded traffic object recognition ofthe present systems and methods maximizes traffic intersectionthroughput and minimizes traffic stoppage of all traffic objectsincluding those which are those which are not detected by conventionalsensors.

FIG. 3 shows another typical traffic light configuration 302 more commonin urban areas having two or more traffic lanes in each direction anddedicated left turn lanes. The dedicated left turn lane signal light 304commonly includes only left turn arrows in the same red yellow and greencolors and orientation as the standard signal lights placed over thethrough-going [straight] lanes (306, 308), hence providing trafficcontrol commands specific to the left turn lane traffic. As shown inFIG. 3, prior conventional traffic intersections are equipped withweight- or inductive loop sensors embedded in the roadways atsignificant initial cost, not to mention substantial ongoing costsrequired to modify or repair such sensors. Basic traffic flow predictioncapability is provided by placement of additional such sensors offset ata distance from the traffic lights sufficient to trigger a signal lightsequence change to accommodate incoming traffic. Several of the newertraffic sensors described previously can be installed at theintersection at costs far lower than those incurred when roadwayreconstruction is required as with prior art.

This figure also illustrates the superiority of aspects of the presentinnovations as shown by a Vehicle1 320 in the center through-going lanewith its left turn signal lights flashing. Having already triggered themost distant offset inductive sensor 310, the vehicle will haveinitiated the changing of traffic light states to the color green,allowing traffic to pass through the intersection, in both through-goingdirections of the vehicle whenever no competing traffic from theorthogonal directions is detected on the assumption that Vehicle1 320will proceed straight through the intersection over straight laneinductive sensor 312. In current systems, it is not until Vehicle1 320crosses left turn lane sensor 310 that the driver's intention to turnleft is detected. This standard traffic flow decision of the prior artis the wrong decision in this case. Full compliance with this driver'sintentions requires the recognition of the driver's signal for making aleft turn, the initiation of a lighting subsequence with a green leftturn arrow combined with red [stop] light indications for through-goingtraffic in the other three directions or an equally-safe subsequenceindicating left turns allowed in Vehicle1's lane and its counterpartleft turn lane directly opposite. The present innovations mayaccommodate traffic light sequence initiation and higher-level neuralnetwork recognition of such drivers' intentions for making optimaltraffic flow decisions in such cases by having turn signal indicationsincorporated in the set traffic objects stored in lower-level neuronobject recognition. An added benefit of this traffic signal controlcapability is training the general public to use their vehicle turnsignal indicator lights to signal their intentions to both the trafficcontrol system and their fellow drivers. While the intention ofplacement of distant sensors (316, 318) is to change the traffic lightstate in advance of incoming traffic, the wrong decision can be madewithout neural network recognition of driver intention. Due to real-timerecognition and decision-making, the neural network selects the optimaltraffic flow control decision immediately after Vehicle1's state isrecognized as having a left turn indicator light flashing.

FIG. 4 shows one of several traffic lights sequences as may beprogrammed into the programmable sequencer logic of traffic lightcontroller, and describing a sequence common to a straightforward,single through-traffic lane intersection also having a dedicated leftturn lane shown previously in the FIG. 2. A value of “1” indicates the“ON” state for the traffic light lens indicated in the left-hand column,wherein the left turn arrow [Red-, Yellow-, Green-LT] may in fact bedifferent states of the left turn arrow lens occupying the fourth, orlowest, position of a single traffic light as shown previously in FIG.2.

The sequence set forth in FIG. 4 describes one beginning with left turntraffic from both opposing directions is allowed first, followed bythrough-going [straight] traffic in those same directions while trafficin the orthogonal directions is stopped. It should be viewed as twoseparate sequences which might be labeled “both left” followed by “bothstraight.” States 1 and 10 can be described as the “ALL STOP” statewhich, in some implementations, is a base state from which anyalternative light sequence can be initiated. The programmable trafficlight sequencer may store all useful sequences needed for any particulartraffic light and direction combination. Examples of other sequences forthis same intersection's traffic for this direction include: a) swappingthese subsequences to allow “both straight” followed by “both left,” b)single left turn and through-traffic from one direction followed by“both straight,” possibly followed by single left turn andthrough-traffic from the other direction, or c) subsets of these, all ofwhich are deployed in prior conventional traffic light control systems.The previously mentioned “FailSafe” mode may use a complete set ofsub-sequences insuring that all traffic lanes from all directions aregiven the minimum specified green light time. Some time interval isprogrammed for the “ALL STOP” state as a boundary between all programmedsubsequences so that any subsequence can be selected after any other andso that the light can be held in the “ALL STOP” in an emergency.

An advantage consistent with one or more aspects of the innovationsherein is the ability to learn and recognize an exhaustive set oftraffic flow patterns from all directions converging on the intersectionand to learn the optimal traffic flow decisions for each case throughthe use of specific neural network training yielding optimal trafficflow results, such as minimal vehicle wait times, maximum vehiclethroughput, maximum passenger-weighted throughput, or vehicle tonnagethroughput. Unique traffic pattern recognition trained to higher-levellayers of the neural network produces unique output which the systemuses to select sequences and programmable timer base values whichmicroprocessor loads into sequencer and timer.

FIG. 5 shows another set of traffic lights sequences typical inmulti-lane intersections as depicted in FIG. 3. Here, states #1 and #12are the base “ALL STOP” states, and sequences might be labeled “singleleft and straight” followed by “opposite left and straight,” with state# 13 indicating the beginning of a third subsequence, a “left andstraight” flow command from one of the two adjacent orthogonaldirections. These are by no means exhaustive set of subsequences and areincluded only for the purposes of illustration and to show salientsequences for particular traffic lane and light situations. They alsoillustrate that any subsequence can be selected from the base “ALL STOP”state, yielding the ability to initiate overall sequences for all signallights facing each direction in an intersection for optimal overalltraffic flow. State #3 shows a blinking green left turn arrow precedingthe yellow “caution” state of that arrow lens for providing next statewarning to drivers in that left turn lane as has been described in priorart. Similarly, state #7 shows the round green lens of the throughtraffic lanes in that same direction also blinking prior to the changeto the yellow “caution” state. The current innovations may provide anadditional level of safety through the training of higher-level neuronsfor speed and distance recognition whereby programmable timer values canbe updated in real time to modify signal light timing for collisionavoidance. The same prediction can be used for traffic light violationdetection and reporting. Both collision avoidance and traffic violationdetection have been described in prior art, but implementations of thecurrent innovations are unique in deploying high-capacity neuralnetworks for such detection and minimal chip solid-stateimplementations, improving performance over prior digital computationalmethods.

FIG. 6 lists examples of neural network training data as a function offour hierarchical neuron layers, according to illustrative systems andmethods. This is by no means a specific detailed implementation; rather,it serves as an example for illustration and discussion purposes only.Detailed characteristics of sensor array data collection and neuralnetwork recognition capabilities are required to define an efficient andoptimal configuration. In systems so equipped, a neural network of verylarge size may store enough individual vehicle image vectors to allowexact match [Match 1] recognition of a very high percentage of vehiclescurrently using the roadways. More common and sufficiently useful forthis application is classification of vehicles commonly used in theautomotive industry such as sedan, truck, or SUV. Detailedclassification of specific types of vehicles such as hybrid or electricvehicles [EV's] may be useful for traffic control decision-making inhigher neural network hierarchical levels. Similarly, high confidencerecognition of motorcycles and bicycles is attained through training ofLayer 1 neurons. The present innovations are far superior in thatregard, allowing all such high-efficiency methods of transportation toinfluence traffic flow decision-making. As discussed previously, audioor frequency specific radio signals can be input and recognized by layer1 neurons to initiate priority override traffic light sequences forERV's or HOV's. Multiple Layer 1 neurons can be employed to recognizethe relative position of each incoming traffic object along with therecognition of the object itself or in conjunction with other neuronstrained to identify specific traffic object types.

For the purposes of distance and position recognition of multipletraffic object types, the neuron training may have several distinctfields within a video sensor where different neurons are trained torecognize any number of traffic object classes in these specificlocations. These neurons are assigned to the same context, forming a“recognition engine” for recognizing that particular object class, inthis case position and distance. Video data streams from right and left,or stereo, cameras as shown in FIG. 2 may be particularly useful forposition and distance recognition.

Other neuron groups are assigned to other recognition engines, orcontexts, and specific categories within contexts for detailed trafficobject recognition. It may be useful to separate significantly differenttraffic object types, such as private vehicles, commercial vehicles, andbicycles into separate recognition engines or contexts since this mayfacilitate higher-level traffic flow decision prioritization. Due to thesize of relative video input fields, it may be useful to assign uniqueneurons to various groups sizes of small traffic objects such asbicycles. For example, the network may be trained to recognize threebicycle group objects: a solo cyclist, two new cyclists, or a group ofthree or more. This may be the optimal method for recognizing cyclists,all within the Layer 1 hierarchy, as opposed to aggregating trafficobject groups in higher neural network layers as in the case of largertraffic objects. This illustrates the experimentation and judgmentrequired for assigning neurons to specific recognition tasks.

For a given recognition context and traffic object category, the controlsystem may input various types of sensory input. For example, aspreviously mentioned, a radar array may provide unique traffic objectsignatures in addition to highly accurate position information. Theseradar data can be input as part of a large input vector to the neuralnetwork array in combination with video and other sensory input or theycould be fed separately into other dedicated neurons of the same contextand category, another illustration of how the system configuration mayvary with the chosen input sensor array or size of the neural networkarray. As noted previously, it may be useful to have separate contextsassigned to dedicated neuron groups for each incoming traffic direction.

Though a vehicle's position may be implicit in specific learned trafficobject recognition neurons, separate neurons may be trained to recognizeposition independent of object class, i.e. are grouped in a specializedcontext for distance and position. These neurons may be trained to firewhen any type of traffic object is recognized as entering specific zonesapproaching the traffic light. By accumulating position data, higherlevel neurons can be trained to recognize relative speed of approachingtraffic to facilitate traffic flow decisions in time. Similarly, theposition among multiple lanes is valuable for higher leveldecision-making. Additional sensor input may also allow recognition ofvehicle occupancy. The recognition of platoon and high occupancy stateof incoming traffic could be used to prioritize and optimize trafficflow throughput on the basis of passengers instead of just the vehicles.Other neurons could be trained to recognize active states of vehicleturn signals which, as described earlier, can improve traffic flowthrough better understanding of the driver's intention.

Layer 2 neuron training in this example generally involves recognitionof multiple traffic objects' characteristics or their relationships withthe traffic intersection and nearby vehicles. Research has shown thatoverall traffic flow is improved when vehicles are grouped into“platoons,” and prior art has described specialized traffic signalcontrollers which force platoon formation and regulate traffic flowaccordingly. The present innovations are superior in this regard sinceLayer 2 recognition automatically detects spontaneously formed platoonswhich can be given traffic signal priority. Similarly, recognition ofvehicle group characteristics from nearby intersections can be used tooptimize traffic flow decision-making over a wider area. Sequences oftraffic recognized in Layer 1 can be accumulated over multiple systemclock cycles to generate recognition of traffic object wait times andrelative distance and velocity of incoming traffic. Top cameras canobserve pedestrian traffic and adjust signal timing as described inprior art when speed and position of individual pedestrians arerecognized as placing them in danger. As mentioned previously,recognition of groups of traffic objects may also be transmitted tonearby similar controllers to optimize traffic flow over a larger area.Unique advantages of aspects of the present innovations may involvehaving all similar controllers operating autonomously without complex,inefficient fixed timings and sequences or centralized control whilestill utilizing information from a wider traffic area.

Examples of layer 3 recognition include higher-level patterns such asaverage speeds of multiple incoming traffic objects, aggregate waittimes, or cumulative passenger occupancy. High-level inputs from sourcessuch as a central network-wide control station may also be included.This is the highest recognition level in this example where combinationsof various incoming traffic are recognized. High-level prioritizationbased on these aggregated traffic object recognitions can beincorporated into the training to regulate traffic flow from alldirections.

Further, according to this illustrative implementation, layer 4 neuronsmay be trained to recognize optimal traffic flow decisions based uponinput from all lower hierarchical layers. Programmable interconnectlogic 130 may route data from the configurable I/O from specific neuronsto specific layer for neurons. Current state information of theprogrammable sequencer 122 and timer module 120 may also providecritical input for Layer 4 neurons deployed to detect potential trafficviolators or traffic collisions. Training algorithms for Layer 4 neuronsmay seek optimum traffic flow decisions yielding minimum wait times forall traffic objects and maximum overall traffic throughput. One or moreLayer 4 neurons is/are trained for each subsequence programmed into thesequencer. Similarly, each subsequence may have two or more timing sets,each of which has one or more Layer 4 neurons trained to recognize theoptimal timing decisions, including a timing hold in the “ALL STOP”state when traffic light violators are detected and this decision canprevent a traffic collision. The microprocessor 108 loads theappropriate command for a specific traffic light subsequence and valuesfor the programmable timer assigned to a particular Layer 4 neuron,effecting the optimal traffic flow decision.

As mentioned previously, the training could force traffic flow decisionswhich optimize passenger throughput of the intersection as may bepreferable in many urban environments. Alternatively, the training mightforce decision-making which optimizes fuel economy as may be preferablein industrial zones. For example, large tractor-trailer rigs with thedriver being the sole passenger may be given priority through someintersections to avoid fuel wasting stoppages. Ironically, the moreefficient modes of transportation such as bicycles, motorcycles,hybrids, EV's, or even HOV's may be stopped to facilitate higherthroughput of higher gross weight vehicle traffic in such a zone. Thus,the high-level traffic flow decision training is specific to eachautonomous traffic light controller. The training can be updatedwhenever additional sensory input or vehicle classes are added to thesystem or when traffic flow priorities are changed. The traffic lightcontroller thereby adapts to the needs of the local traffic area overtime without requiring hardware modifications to the controller itself.

Further, in other implementations, the training (training algorithms)may force decision-making which optimizes traffic flow in a particulardirection. For example, the training may operate as a function of aspecific timing regime, such as a timing regime that altersweighting/operation as a function of a conservative timing regime and/oran aggressive timing regime. Further, an intersection may have a timingregime that is conservative in one or more directions or lanes andaggressive in others.

FIG. 7 summarizes an illustrative process flow for initial systemconfiguration in stepwise fashion, consistent with one or more aspectsof the innovations herein. Referring to FIG. 7, a representativefour-step process is depicted including various optional aspects, thoughno process decision branching is shown. FIG. 7 includes various mainelements shown on the left side which may be achieved in a number ofways, while other, more optional elements, shown within dashed-lineboxes on the right side, represent detailed steps according to variousimplementations, using hardware and software available in the timeframeof the disclosure of the present invention.

According to the exemplary implementation of FIG. 7, an initial step mayinclude processing system architecture information, 702. Desired outputsor results of such processing, which may comprise various manual and/orcomputerized information analysis, may include having workingdefinition(s) of a traffic intersection signal light and sensor hardwareand/or a general plan for neural network array implementation to enablesystem architecture configuration, at 704. Further, in someimplementations, these parameters may be modeled outside the ambit ofthe present innovations, with definition(s) of system architecture beingdelivered in some standardized format, hence such processing of systemarchitecture information is shown as being optional. Detailed steps ofthis process flow these are shown on the right, starting with definitionof traffic light sequences and associated timer values for this specifictraffic light configuration. Next is clear characterization andassignment of sensor array devices, wherein capabilities of sensingdevices are quantified and mapped to recognition contexts and categoriesfor which the neural network will be trained. Details and categories forrecognize traffic objects are constrained by the size of the neuralnetwork implementation and may include any or all of previouslydescribed characteristics, including details of traffic objects that mayinfluence prioritization such as HOV or ERV designation. In addition torecognition of individual traffic objects and their associated classes,other neuron groups may be assigned to detect distance and positionrelative to the intersection. Still other neurons may be assigned torecognize custom HOV/ERV priority inputs.

Higher-level neuron layers are trained to detect traffic object statussuch as vehicle grouping and speed. This requires concurrent definitionof interconnections between hierarchical neuron layers. For example, thedetection of traffic objects and specific distance/position zones may beaggregated over successive neural network recognition cycles forrecognition of individual or group traffic object speeds and wait timesfor stopped traffic objects. Information developed as a function of thenumber of cycles required for a change in position of traffic objects orgroups may be provided as input to neural network recognition enginesfor these traffic object state classifications. Different neuronsconfigured to aggregate lower-level recognitions over varying ranges ofrecognition cycles yield recognition of different categories of trafficobject speeds and wait times.

Highest level neuron layers may be configured to recognize input vectorsfrom lower layers, resulting in classification outputs representingspecific traffic flow decisions. The number of these neurons allocatedto these tasks may be assigned as a function of the number of trafficlight sequences and related timings required for specific hardwareconfiguration of the traffic signal light.

A final aspect of processing system architecture information 702 mayinvolve allocating sets of reserve neurons which can be used to enableadditions or upgrades to system hardware or for retraining sets ofneurons during real-time operation of the traffic light controller usingcurrent neural network training sets.

The illustrative step of configuring system architecture, 704, may beimplemented in a number of ways, including configurations involving ahardware system containing system software and logic configurations andmask-programmable logic devices. One exemplary implementation shown onthe right involves writing such configuration into programmable hardwareas in the preferred embodiment of the present invention. System firmwaremay be downloaded into nonvolatile memory and programmable logic such asthat used for the programmable sequencer, the configurable I/O, theprogrammable interconnect logic, the DSP configuration, the system clockcontroller, and/or the communications interface module. Decodingspecific standardized communication interfaces or custom ERV/HOV signalsmay be achieved by microprocessor executed firmware or programmablelogic within the communications interface module.

Processes of neural network training, 706, may be achieved bydownloading externally acquired training, or on site learning requiredby specific local system, or a combination of the two. Real-time neuralnetwork training required on-site may involve successive training of anumber of neurons until each targeted traffic object, traffic objectstate, and optimal decision is recognized with sufficient distinctionfrom alternative classifications. The training process may be iterated,i.e. performed until all targeted recognitions of required elements isachieved and all counter examples are rejected. In some implementations,when a training set is presumed to be complete, the system may bevalidated by running the traffic light controller in “FailSafe” modewhile collecting data on the autonomous decision-making of the highestlevel neuron layer. These results may be compared with the “correct”training examples, and the training may be repeated until these datamatch.

After neural network training, an autonomous operation startup processmay be performed, 708. Various optional steps that may be performed insuch process may include initially checking one or more of the properfunction of sensor arrays, the communication module, the basic signallight sequencer and/or the timer function. Next, the system may berestarted in “FailSafe” mode, e.g. for some predetermined startup time,after which autonomous traffic decision flow control determined by theneural network may be enabled.

Overall exemplary system architecture, here, may first define a completeset of light sequences and timer module data sets specific to thetraffic light configuration. Next a complete training data set of allinput from the sensor arrays may be captured and stored. The contextsand category labeling for each traffic object class, traffic groups,position and distance zones must be defined, followed by definitions fortraffic flow state recognition such as incoming traffic speed andindividual and cumulative wait times for stopped traffic. Where theoutput of lower level neurons provides input to higher-level neurons,the required I/O configurations may be programmed into the programmableI/O PLA as too may the associated inter-layer connections betweenneurons hierarchical levels be programmed into the programmableinterconnect logic PLA. The highest neural network layer may have one ormore neurons assigned to each traffic flow decision represented by thematrix of each programmable sequencer entry point and associatedprogrammable timer data sets. The number of neurons required for eachtraffic object, traffic state, and traffic flow decision may beallocated within each neural network layer, along with a number ofreserved neurons for all recognition or decision processes selected forreal-time training during normal live operation. Further, DSP modulesmay be configured to convert incoming data streams to match the neuralnetwork input vector size. The communication module may be configuredfor standard network communication protocols and/or for detection ofspecific high-priority ERV or HOV signals. In certain implementations,the PLA of the clock generator may be programmed for each specializedtiming signal, from the base processor and system bus clock rates tothose for the neural network recognition engines, the programmablesequencer timing input, the DSPs, various timings of the communicationmodule, and various data capture cycles of different traffic sensordevices.

Where the traffic intersection closely matches that of a known existingimplementation, some or all of the neural network training data setsfrom such remote locations may be programmed directly into the neuralnetwork array. Otherwise, the neuron training may be accomplished at thespecific intersection of the present traffic light control system.Again, once complete training of the entire neural network isaccomplished, the traffic control system may begin operation in thedefault “FailSafe” mode while data is collected for traffic flowdecisions indicated by the top-level neuron layer so that intendedfunction may be validated. Following system validation, the trafficcontrol system may be set to operate in full autonomous mode accordingto the neural network training.

FIG. 8 is a diagram of an illustrative process flow for neural networkarray training. The training data sets from the traffic sensor arraysmay first undergo an initial layer training 802, wherein they arepresented to the lowest layer neurons, which may be trained to recognizeeach defined traffic object, traffic object group, and/orposition/distance zone. In a recognitions processing step 804, data thatis not recognized or classified 806 may be returned for furtherprocessing, while recognized data 808 is passed through for secondaryhandling 810 such as storage and processing associated with arecognition mode. The required number of neurons may be activated untilthe complete set of these recognitions is accomplished. It may besufficient for most traffic flow decisions to have overlapping influencefields of various neurons which can yield Match 2, or uncertain,classifications, since the recognition of any traffic object, especiallyin the case of a solo traffic object, can still produce the correcttraffic flow decision at the highest level. The accuracy of recognitionis determined by number of available neurons relative to the number ofexample input vectors to be recognized.

After lowest layer neurons are trained, they are set to normaloperational recognition mode (see 810) and presented with the sametraining data set they were trained with, while the next higher neuronlevel is put into training mode, referred to as intermediate layertraining 812. The next level neurons are progressively activated andtrained 814, 816 until the defined set of classifications for that levelare learned 818. This process is iterated 820, 822, 824 for eachintermediate layer when there is more than one such layer. After thelast intermediate layer is processed 826, these layers are put intorecognition mode for the training of the highest neuron layer 828 Thesame example inputs used to train the lower levels may be used again intraining the highest layer, though in some implementations this trainingmay be focused on the highest level traffic flow decisions. Suchtraining components, such as training software, may use weightingsteaching neurons to select light sequences and timings which maximizeoverall traffic throughput and minimize traffic object wait times, butmay also use other considerations such as vehicle class or passengeroccupancy to prioritize specific traffic objects as describedpreviously. Further, observed data of actual vehicle wait times may bebe monitored by the microprocessor and used to alter the training inputson the training software on each pass until a minimum is reached for anygiven traffic pattern. The traffic light sequence and timing resultingin such optimal solution may be selected as the “correct” decision, andthe location of the neuron trained to fire in a specific trafficsituation may be the information decoded by the microprocessor to selectthat specific sequence and timing. One or more top layer neuron may beactivated and trained 830 for each combination of programmable sequencerand programmable timer settings which represent traffic flow decisions.Here, for example, such training may entail processing, via aniterative/loop process 832, to provide a complete, unique decision setfor each priority input and sequence/timing option. Finally, eachtraining data set for every context, i.e. recognition engine, withineach neural network hierarchical layer may be stored 836, such as insystem firmware or in an off-line remote location, and final validationtesting may be performed. Once this processing is complete, the data isavailable for subsequent system maintenance or for use in similarsystems at other locations.

FIG. 9 is an illustrative software flowchart depicting exemplary systemstartup and operation features, consistent with one or more aspects ofthe innovations herein. In the exemplary diagram of FIG. 9, the systemstarts up in FailSafe mode 904 following initial power up and systembootstrapping 902. Neural network recognition 906 may be enabledfollowing successful diagnostic testing of sensor arrays and thecommunication module. The programmable interconnect logic may beconfigured to route, at 908 and 916, only Match 1/2 flags from highestlevel traffic flow decision neurons to the microprocessor. Match 1, oridentified, classifications are scanned first, beginning with highestpriority decisions 908 such as those generated by ERV or HOV specialsignals or traffic flow direction from the remote command center. Ofcourse, neurons assigned to the context for detection of potentialcollisions may be included among these highest priority Match 1classifications to be monitored. Other Match 1 classifications arespecific optimal traffic flow decisions which are initiated directly tothe programmable sequencer and timer. When decisions are confirmed bythe neurons 910, 920, the associated programmable sequencer and timersettings may be loaded 912, 922. The microprocessor may scan the flaggeduncertain Match 2 classifications (918) next, at 926 and 928. Where twoor more neurons indicate the same traffic flow decision 930, theassociated programmable sequencer and timer settings are loaded 932.Otherwise 934, the microprocessor determines which decision is mostcompatible with the current traffic signal state and implements thatdecision 936, since the uncertain classification implies that eitherdecision is acceptable. As with processing the certain Match 1 decisions924, the control software may then branch back to scanning for new Match1 decisions 906.

FIGS. 10A-10B are a diagram and a flowchart, respectively, ofhigher-level recognition processing as performed by a Neural Network,consistent with one or more aspects of the innovations herein. FIG. 10Ais a diagram showing an exemplary traffic state near a trafficintersection which illustrates exemplary higher-level recognitionaspects. FIG. 10B sets forth an illustrative, generalized process ofsuch higher-level recognition processing. Referring to FIG. 10B, suchgeneralized process may include obtaining positional data 1032 (e.g.,obtaining positional information using low level neurons assigned torecognize objects as being present within defined zones), performingaggregation of this data 1034 (e.g., aggregating positional stateinformation from low level neurons over multiple classification cyclesby higher level neurons to recognize patterns or information, such asarrangements and/or relative speeds of traffic objects within monitoredpositional zones), and further processing the aggregated informationover multiple classification cycles 1036 (e.g., aggregating multipletraffic pattern and/or relative velocity states from multipleintermediate level neurons over multiple classification cycles byhighest level neurons to recognize global traffic state and/orassociated optimal traffic flow decision(s)).

Turning back to the illustrative implementation of FIG. 10A, analphanumeric array, 1002, represents positional zones described earlierin the vicinity of traffic signal light 1004. Traffic object group 1006represents a self organized “platoon” of traffic objects, such as aconvoy of cars or other vehicles. In this diagram, which shows onedetailed example of the generalized process of FIG. 10B, specificlower-level neurons may be trained to recognize traffic objects as beingpresent in each positional zone which are designated in this drawing bycoordinates ranging from A1 through A6 to N1 through N4. The trafficplatoon 1006 has traffic objects in zones N2, M1, L3, K1, and J2. Whilesome lower-level neurons may be assigned to specific traffic object typerecognition, other low-level neurons may be assigned to detect trafficobjects of any type as being present in specific zones. Higher-levelneurons aggregate the these recognitions into recognitions of trafficobject groups and states. For example, the traffic object shown in zoneN2 will be recognized over successive neural network classificationcycles to have progressed through zones M2, L2, K2, J2, and so on, if itcontinues in a straight line. The number of recognition cycles consumedduring this progression translates directly to vehicle speed.Aggregation of such progressions yields classification varying ranges oftraffic objects speed. Further, specific neurons can be trained torecognize specific speed ranges, which, in turn, can provide input toeven higher level neurons for traffic light flow decisions, collisionavoidance, or traffic light violation prediction.

In some implementations, other sets of neurons can be trained to detectone or more sizes of traffic object platoons, which would be prioritizedover single traffic objects when processed by higher-level neurons fortraffic flow decisions. Similarly, other sets of neurons may be trainedto recognize stopped traffic. For example, traffic groups 1008, 1010,and 1012 may be recognized as stopped traffic by higher-level neuronsdetecting traffic objects occupying the same position zone over one ormore neural network classification cycles, wherein the total number ofcycles is proportional to traffic object wait times. Different sets ofneurons may be trained to recognize any important pattern, so in thisexample an additional set of neurons can be assigned to recognizecumulative wait times, the sum of stopped traffic objects multiplied bytheir stoppage times. A training input that may be important for theneural network is to minimize cumulative wait times, so a properlytrained highest level set of traffic flow control neurons wouldrecognize in this example that these groups should be prioritized in theorder of 1008, followed by 1010, then 1012, based on the number ofdetected waiting traffic objects assuming similar average wait times.Individual traffic objects 1014, 1016, and 1018, may add weight to theprioritization of traffic flow from their respective directions, thoughneural network training algorithms consistent with the presentdisclosure may prioritize these at a lower level for two reasons: 1)individual traffic objects have less weight than traffic object groups,and 2) these three traffic objects are all in right-hand lanes which maypossibly pass through the intersection with right-hand turns that do notrequire changes to the traffic signal light. Traffic objects 1020 and1022 may, in general, be irrelevant to traffic flow decisions. In fact,some implementations may ignore all traffic object position zones fortraffic lanes exiting the intersection, including zones N3-N4 to I3-I4and A5-6 to C5-6 in this example. The capabilities of any implementationmay be determined as a function of the size of the neural networkimplementation. So, for example, these same zones which may be excludedfor a very wide range of traffic scenarios could prove useful fordetecting dangerous “outlier” circumstances such as a vehicleapproaching the intersection in the opposite direction of the legalconvention.

Further, according to some implementations, the highest level neuralnetwork layer interpreting the data recognized by the lower-level neuronlayers, here, would recognize a specific traffic flow optimizationdecision. In such cases, the next traffic flow sequence selected fromany compatible prior state or “All Stop” state would be a sequenceallowing a left turn by traffic group 1008 and a green light for throughtraffic passage by traffic group 1006.

Additionally, the innovations herein may be achieved via implementationswith differing or entirely different components, beyond the specificcomponents and/or circuitry set forth above. With regard to such othercomponents (e.g., circuitry, computing/processing components, etc.)and/or computer-readable media associated with or embodying the presentinnovations, for example, aspects of the innovations herein may beimplemented consistent with numerous general purpose or special purposecomputing systems or configurations. Various exemplary computingsystems, environments, and/or configurations that may be suitable foruse with the innovations herein may include, but are not limited to,various clock-related circuitry, such as that within personal computers,servers or server computing devices such as routing/connectivitycomponents, hand-held or laptop devices, multiprocessor systems,microprocessor-based systems, set top boxes, smart phones, consumerelectronic devices, network PCs, other existing computer platforms,distributed computing environments that include one or more of the abovesystems or devices, etc.

In some instances, aspects of the innovations herein may be achieved vialogic and/or logic instructions including program modules, executed inassociation with the circuitry, for example. In general, program modulesmay include routines, programs, objects, components, data structures,etc. that perform particular tasks or implement particular control,delay or instructions. The inventions may also be practiced in thecontext of distributed circuit settings where circuitry is connected viacommunication buses, circuitry or links. In distributed settings,control/instructions may occur from both local and remote computerstorage media including memory storage devices.

Innovative circuitry and components herein may also include and/orutilize one or more type of computer readable media. Computer readablemedia can be various media that is resident on, associable with, or canbe accessed by such circuits and/or computing components. By way ofexample, and not limitation, computer readable media may comprisecomputer storage media and communication media in tangible format(s),though does not encompass transitory media. Computer storage mediaincludes volatile and nonvolatile, removable and non-removable mediaimplemented in any method or technology for storage of information suchas computer readable instructions, data structures, program modules orother data. Computer storage media includes, but is not limited to, RAM,ROM, EEPROM, flash memory or other memory technology, CD-ROM, digitalversatile disks (DVD) or other optical storage, magnetic tape, magneticdisk storage or other magnetic storage devices, or any other mediumwhich can be used to store the desired information and can accessed bycomputing component. Communication media may comprise computer readableinstructions, data structures, program modules or other data embodyingthe functionality herein in tangible (non-transitory) forms.Combinations of the any of the above are also included within the scopeof computer readable media.

In the present description, the terms component, module, device, etc.may refer to any type of logical or functional circuits, blocks and/orprocesses that may be implemented in a variety of ways. For example, thefunctions of various circuits and/or blocks can be combined with oneanother into any other number of modules. Each module may even beimplemented as a software program stored on a tangible memory (e.g.,random access memory, read only memory, CD-ROM memory, hard disk drive)to be read by a central processing unit to implement the functions ofthe innovations herein. Also, the modules can be implemented as hardwarelogic circuitry implementing the functions encompassed by theinnovations herein. Finally, the modules can be implemented usingspecial purpose instructions (SIMD instructions), field programmablelogic arrays or any mix thereof which provides the desired levelperformance and cost.

As disclosed herein, implementations and features consistent with thepresent inventions may be implemented through computer-hardware,software and/or firmware. For example, the systems and methods disclosedherein may be embodied in various forms including, for example, a dataprocessor, such as a computer that also includes a database, digitalelectronic circuitry, firmware, software, or in combinations of them.Further, while some of the disclosed implementations describe componentssuch as software, systems and methods consistent with the innovationsherein may be implemented with any combination of hardware, softwareand/or firmware. Moreover, the above-noted features and other aspectsand principles of the innovations herein may be implemented in variousenvironments. Such environments and related applications may bespecially constructed for performing the various processes andoperations according to the present innovations or they may include ageneral-purpose computer or computing platform selectively activated orreconfigured by code to provide the necessary functionality. Theprocesses disclosed herein are not inherently related to any particularcomputer, network, architecture, environment, or other apparatus, andmay be implemented by a suitable combination of hardware, software,and/or firmware. For example, various general-purpose machines may beused with programs written in accordance with teachings of theinnovations herein, or it may be more convenient to construct aspecialized apparatus or system to perform the required methods andtechniques.

Aspects of the method and system described herein, such as the logic,may be implemented as functionality programmed into any of a variety ofcircuitry, including programmable logic devices (“PLDs”), such as fieldprogrammable gate arrays (“FPGAs”), programmable array logic (“PAL”)devices, electrically programmable logic and memory devices and standardcell-based devices, as well as application specific integrated circuits.Some other possibilities for implementing aspects include: memorydevices, microcontrollers with memory (such as EEPROM), embeddedmicroprocessors, firmware, software, etc. Furthermore, aspects may beembodied in microprocessors having software-based circuit emulation,discrete logic (sequential and combinatorial), custom devices, fuzzy(neural) logic, quantum devices, and hybrids of any of the above devicetypes. The underlying device technologies may be provided in a varietyof component types, e.g., metal-oxide semiconductor field-effecttransistor (“MOSFET”) technologies like complementary metal-oxidesemiconductor (“CMOS”), bipolar technologies like emitter-coupled logic(“ECL”), polymer technologies (e.g., silicon-conjugated polymer andmetal-conjugated polymer-metal structures), mixed analog and digital,and so on.

It should also be noted that the various logic and/or functionsdisclosed herein may be enabled using any number of combinations ofhardware, firmware, and/or as data and/or instructions embodied invarious machine-readable or computer-readable media, in terms of theirbehavioral, register transfer, logic component, and/or othercharacteristics. Computer-readable media in which such formatted dataand/or instructions may be embodied include, but are not limited to,non-volatile storage media in various forms (e.g., optical, magnetic orsemiconductor storage media), though do not encompass transitory media.

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense as opposed to anexclusive or exhaustive sense; that is to say, in a sense of “including,but not limited to.” Words using the singular or plural number alsoinclude the plural or singular number respectively. Additionally, thewords “herein,” “hereunder,” “above,” “below,” and words of similarimport refer to this application as a whole and not to any particularportions of this application. When the word “or” is used in reference toa list of two or more items, that word covers all of the followinginterpretations of the word: any of the items in the list, all of theitems in the list and any combination of the items in the list.

Although certain presently preferred implementations of the presentinnovations have been specifically described herein, it will be apparentto those skilled in the art to which the inventions pertain thatvariations and modifications of the various implementations shown anddescribed herein may be made without departing from the spirit and scopeof the inventions herein. Accordingly, it is intended that theinventions be limited only to the extent required by the appended claimsand the applicable rules of law.

1. A method for processing traffic information, the method comprising:receiving data regarding travel of vehicles associated with anintersection; using neural network technology to recognize types oftraffic; using the neural network technology to recognize states of thetraffic; using the neural network technology to memorize optimal trafficflow decisions as a function of prior experience; and using the neuralnetwork technology to achieve efficient traffic flow via recognition ofthe optimal traffic flow decisions.
 2. (canceled)
 3. The method of claim1 further comprising performing video image processing optimized forneural network recognition and/or control.
 4. The method of claim 1,further comprising enhancing the traffic type recognition using existinginfrastructure. 5.-8. (canceled)
 9. The method of claim 1, furthercomprising enhancing the traffic type recognition using an array havinga plurality of sensor inputs beyond inputs from video sensors andtraditional physical infrastructure.
 10. The method of claim 9, whereinthe system includes a sensor array that provides a plurality of inputvectors in addition to inputs from basic video capture available at theintersection.
 11. The method of claim 9 wherein the other sensor inputsinclude a traffic radar input providing enhanced capability fromtransmission of various data types.
 12. The method of claim 11 whereinthe traffic radar input comprises: detailed radar signatures of detectedtraffic objects independent from adverse weather or light conditions forrecognition of vehicle position and type; and radar measurement ofvelocities of incoming traffic objects; wherein prediction of trafficflow is enhanced for higher level signal light control decisionsincluding prioritization of incoming traffic and collision-avoidancesignal light hold.
 13. The method of claim 9 wherein the other sensorinputs include an infrared traffic input providing enhanced capabilityfrom transmission of various data types.
 14. (canceled)
 15. The methodof claim 1, further comprising: processing new sensory inputs;retraining previously deployed neurons to take advantage of the newsensors; wherein recognition of the traffic types is enhanced viaadapting their recognition to utilize the new inputs.
 16. The method ofclaim 1, further comprising: processing new sensory inputs; utilizingadditional, reserve neuron capacity that was initially included in thesystem, wherein the reserve neuron capacity is subsequently trained toutilize the new inputs.
 17. The method of claim 16 further comprising:improving recognition specificity through the combination of prior andnewly added recognition from the new sensory input.
 18. (canceled) 19.The method of claim 1, further comprising: utilizing/deploying 2 or morelayers of neural network neuron storage elements for unique recognition,classification, and/or traffic flow decision-making tasks.
 20. Themethod of claim 19 wherein the unique recognition includes feeding fromthe low level to the higher levels.
 21. The method of claim 20 wherein,in the step of feeding from the low level to the high level, assignedneurons are: trained to aggregate total numbers of incoming trafficobjects; and trained to recognition position over fixed time sequencesresulting in recognition of relative velocity.
 22. The method of claim19 wherein the classification includes compiling aggregate trafficinformation as a function of neurons assigned to first-level dataincluding one or more of: processing weight/disparity of traffic withineach zone; processing data regarding vehicle occupancy; and/or usingcomposite weighting of all such inputs for higher level traffic flowdecision-making.
 23. The method of claim 19 wherein the traffic flowdecision-making includes state information inputs from nearby similartraffic controllers comparable with intermediate level traffic staterecognition at the local controller which are combined at higher levelsto optimize traffic flow decision-making at the local traffic signallight. 24.-27. (canceled)
 28. The method of claim 1, wherein neuralnetwork recognition is deployed to recognize various standardizedhigh-priority vehicle signal types which is used to initiate requiredauthentication and prioritize traffic flow decisions. 29.-45. (canceled)46. The method of claim 1, wherein the system or method is capable ofrecognizing driver intentions without use of communication/transmissionsbetween the vehicle and the signal light.
 47. The method of claim 1,wherein digital logic and/or communication devices are included toprovide an output of current- and next-state information for use bysimilar, nearby controllers
 48. The method of claim 1, wherein digitallogic is included to provide an output and/or warning of current- andnext-state information configured to be utilized by in-vehiclecommunication devices, as known, that transmit information to/from thetraffic signal and/or warning devices. 49.-69. (canceled)
 70. A methodfor processing traffic information, the method comprising: receivingdata regarding travel of vehicles associated with an intersection; usingneural network technology to recognize types and/or states of traffic;using the neural network technology to memorize optimal traffic flowdecisions as a function of experience information; and using the neuralnetwork technology to achieve efficient traffic flow via recognition ofthe optimal traffic flow decisions. 71.-98. (canceled)
 99. The method ofclaim 1 wherein the traffic comprises automobile traffic.
 100. A methodfor providing highly accurate visual recognition of all types oftraffic, the method comprising: receiving data regarding passage ofvehicles through an intersection; using neural network technology torecognize types of traffic; using neural network technology to recognizestates of the traffic; using neural network technology to memorizesequences determined to be optimal traffic flow decisions as a functionof prior experience; and using neural network technology to achievehighest efficiency traffic flow via recognition of conditions associatedwith the memorized optimal traffic flow decisions.
 101. The method ofclaim 100 wherein the intersection includes one or more traffic lanesinvolving different traffic types wherein traffic flow decisions anddirectives include optimization and/or safe operation for all of thedifferent traffic types.
 102. The method of claim 100 wherein certaintraffic in the intersection is constrained to fixed lanes and lanechange decisions, wherein traffic flow decisions and directives includeoptimization and safe operation for the certain traffic along with allother impending traffic types associated with one or more trafficintersections.
 103. The method of claim 102 wherein traffic lanescomprise traffic routing given by conventions defined for each traffictype, with intersections at each location where traffic objects maytransition from one traffic lane to another wherein traffic flowdecisions and directives include optimization and safe operation forsaid traffic along with all other impending traffic types associatedwith one or more traffic intersections.
 104. The method of claim 100wherein the traffic comprises automobile traffic.